Feedback Loop Failure Modes in Medical Diagnosis: How Biases Can Emerge and Be Reinforced.

BACKGROUND: Medical diagnosis in practice connects to research through continuous feedback loops: Studies of diagnosed cases shape our understanding of disease, which shapes future diagnostic practice. Without accounting for an imperfect and complex diagnostic process in which some cases are more likely to be diagnosed correctly (or diagnosed at all), the feedback loop can inadvertently exacerbate future diagnostic errors and biases. FRAMEWORK: A feedback loop failure occurs if misleading evidence about disease etiology encourages systematic errors that self-perpetuate, compromising future diagnoses and patient care. This article defines scenarios for feedback loop failure in medical diagnosis. DESIGN: Through simulated cases, we characterize how disease incidence, presentation, and risk factors can be misunderstood when observational data are summarized naive to biases arising from diagnostic error. A fourth simulation extends to a progressive disease. RESULTS: When severe cases of a disease are diagnosed more readily, less severe cases go undiagnosed, increasingly leading to underestimation of the prevalence and heterogeneity of the disease presentation. Observed differences in incidence and symptoms between demographic groups may be driven by differences in risk, presentation, the diagnostic process itself, or a combination of these. We suggested how perceptions about risk factors and representativeness may drive the likelihood of diagnosis. Differing diagnosis rates between patient groups can feed back to increasingly greater diagnostic errors and disparities in the timing of diagnosis and treatment. CONCLUSIONS: A feedback loop between past data and future medical practice may seem obviously beneficial. However, under plausible scenarios, poorly implemented feedback loops can degrade care. Direct summaries from observational data based on diagnosed individuals may be misleading, especially concerning those symptoms and risk factors that influence the diagnostic process itself. HIGHLIGHTS: Current evidence about a disease can (and should) influence the diagnostic process. A feedback loop failure may occur if biased "evidence" encourages diagnostic errors, leading to future errors in the evidence base.When diagnostic accuracy varies for mild versus severe cases or between demographic groups, incorrect conclusions about disease prevalence and presentation will result without specifically accounting for such variability.Use of demographic characteristics in the diagnostic process should be done with careful justification, in particular avoiding potential cognitive biases and overcorrection.

Moving outside the board room: A proof-of-concept study on the impact of walking while negotiating.

PURPOSE: Negotiation is a consequential activity that can exacerbate power differentials, especially for women. While traditional contexts can prime stereotypical gender roles and promote conditions that lead to performance differences, these can be mitigated by context shifts. This proof-of-concept study explores whether an easy-to-apply context shift, moving from seated indoors to walking outside, can help improve the quality of negotiated interactions. Here we examine walking's effects on negotiation and relational outcomes as well as experienced emotions, moderated by gender. DESIGN: Same-gender pairs were randomly assigned to either sitting or walking as either candidate or recruiter negotiating a job offer. PARTICIPANTS: Eighty-one pairs of graduate students or community members participated: sitting pairs: 27 women, 14 men; walking pairs: 23 women, 17 men. INTERVENTION: Participants negotiated either while seated (across from each other) or walking (side by side along a path). MEASURES: We measured: negotiation performance (total points) and outcome equity (difference between negotiating party points); subjective outcomes of positive emotions, negative emotions, mutual liking, and mutual trust. With mixed effects models, we tested main effects of condition, gender, and interaction of condition x gender. RESULTS: Relative to sitting, walking was associated with: increased outcome equality for women, but decreased for men (B = 3799.1, SE = 1679.9, p = .027); decreased negative emotions, more for women than men (IRR = .83, 95% CI:[.69,1.00], p = .046); and greater mutual liking for both genders (W = 591.5, p-value = 0.027). No significant effects were found for negotiation point totals, positive emotions, or mutual trust. CONCLUSION: This study provides a foundation for investigating easy-to-implement changes that can mitigate stereotyped performance differences in negotiation.

Assignment-Control Plots: A Visual Companion for Causal Inference Study Design.

An important step for any causal inference study design is understanding the distribution of the subjects in terms of measured baseline covariates. However, not all baseline variation is equally important. We propose a set of visualizations that reduce the space of measured covariates into two components of baseline variation important to the design of an observational causal inference study: a propensity score summarizing baseline variation associated with treatment assignment, and prognostic score summarizing baseline variation associated with the untreated potential outcome. These assignment-control plots and variations thereof visualize study design trade-offs and illustrate core methodological concepts in causal inference. As a practical demonstration, we apply assignment-control plots to a hypothetical study of cardiothoracic surgery. To demonstrate how these plots can be used to illustrate nuanced concepts, we use them to visualize unmeasured confounding and to consider the relationship between propensity scores and instrumental variables. While the family of visualization tools for studies of causality is relatively sparse, simple visual tools can be an asset to education, application, and methods development.

Interruptive Electronic Alerts for Choosing Wisely Recommendations: A Cluster Randomized Controlled Trial.

OBJECTIVE: To assess the efficacy of interruptive electronic alerts in improving adherence to the American Board of Internal Medicine's Choosing Wisely recommendations to reduce unnecessary laboratory testing. MATERIALS AND METHODS: We administered 5 cluster randomized controlled trials simultaneously, using electronic medical record alerts regarding prostate-specific antigen (PSA) testing, acute sinusitis treatment, vitamin D testing, carotid artery ultrasound screening, and human papillomavirus testing. For each alert, we assigned 5 outpatient clinics to an interruptive alert and 5 were observed as a control. Primary and secondary outcomes were the number of postalert orders per 100 patients at each clinic and number of triggered alerts divided by orders, respectively. Post hoc analysis evaluated whether physicians experiencing interruptive alerts reduced their alert-triggering behaviors. RESULTS: Median postalert orders per 100 patients did not differ significantly between treatment and control groups; absolute median differences ranging from 0.04 to 0.40 for PSA testing. Median alerts per 100 orders did not differ significantly between treatment and control groups; absolute median differences ranged from 0.004 to 0.03. In post hoc analysis, providers receiving alerts regarding PSA testing in men were significantly less likely to trigger additional PSA alerts than those in the control sites (Incidence Rate Ratio 0.12, 95% CI [0.03-0.52]). DISCUSSION: Interruptive point-of-care alerts did not yield detectable changes in the overall rate of undesired orders or the order-to-alert ratio between active and silent sites. Complementary behavioral or educational interventions are likely needed to improve efforts to curb medical overuse. CONCLUSION: Implementation of interruptive alerts at the time of ordering was not associated with improved adherence to 5 Choosing Wisely guidelines. TRIAL REGISTRATION: NCT02709772.

Traditional Heart-Healthy Diet and Medication Adherence in the Norton Sound Region: An 18-Month Telehealth Intervention.

INTRODUCTION: Innovations are needed for preventing cardiovascular disease (CVD) and for reaching diverse communities in remote regions. The current study reports on a telemedicine-delivered intervention promoting a traditional heart-healthy diet and medication adherence with Alaska Native men and women residing in the Norton Sound region of Alaska. METHODS: Participants were 299 men and women with high blood pressure or high cholesterol smoking daily who were randomized to receive telemedicine-delivered counseling and printed materials on diet and medication adherence or on smoking and physical activity. Intervention contacts were at baseline and 3-, 6-, and 12-months follow-up, with a final assessment at 18 months. Nutrition outcomes were the ratio of heart-healthy foods and traditional heart-healthy foods relative to all foods reported on a 34-item food frequency questionnaire. Recent and typical adherence for heart medications were self-reported. RESULTS: Intervention effects were significant for the heart-healthy foods ratio at 6 months only (p = 0.014) and significant for the traditional heart-healthy foods ratio at 6 months only for those aged 47+ (p = 0.031). For recent and typical medication adherence, there were no significant group differences by time. DISCUSSION: In a remote region of Alaska, telemedicine proved feasible and acceptable for engaging Alaska Native men and women in counseling on CVD risk behaviors. The findings indicate that more touchpoints may be necessary to impart comprehensive lasting change in heart-healthy eating patterns. Medication adherence group differences were not significant; however, medication adherence was high overall.

Assessment of Prevalence of Adolescent Patient Portal Account Access by Guardians.

Importance: Patient portals can be configured to allow confidential communication for adolescents' sensitive health care information. Guardian access of adolescent patient portal accounts could compromise adolescents' confidentiality. Objective: To estimate the prevalence of guardian access to adolescent patient portals at 3 academic children's hospitals. Design, Setting, and Participants: A cross-sectional study to estimate the prevalence of guardian access to adolescent patient portal accounts was conducted at 3 academic children's hospitals. Adolescent patients (aged 13-18 years) with access to their patient portal account with at least 1 outbound message from their portal during the study period were included. A rule-based natural language processing algorithm was used to analyze all portal messages from June 1, 2014, to February 28, 2020, and identify any message sent by guardians. The sensitivity and specificity of the algorithm at each institution was estimated through manual review of a stratified subsample of patient accounts. The overall proportion of accounts with guardian access was estimated after correcting for the sensitivity and specificity of the natural language processing algorithm. Exposures: Use of patient portal. Main Outcome and Measures: Percentage of adolescent portal accounts indicating guardian access. Results: A total of 3429 eligible adolescent accounts containing 25 642 messages across 3 institutions were analyzed. A total of 1797 adolescents (52%) were female and mean (SD) age was 15.6 (1.6) years. The percentage of adolescent portal accounts with apparent guardian access ranged from 52% to 57% across the 3 institutions. After correcting for the sensitivity and specificity of the algorithm based on manual review of 200 accounts per institution, an estimated 64% (95% CI, 59%-69%) to 76% (95% CI, 73%-88%) of accounts with outbound messages were accessed by guardians across the 3 institutions. Conclusions and Relevance: In this study, more than half of adolescent accounts with outbound messages were estimated to have been accessed by guardians at least once. These findings have implications for health systems intending to rely on separate adolescent accounts to protect adolescent confidentiality.

stratamatch: Prognostic Score Stratification Using a Pilot Design.

In a block-randomized controlled trial, individuals are subdivided by prognostically important baseline characteristics (e.g., age group, sex, or smoking status), prior to randomization. This step reduces the heterogeneity between the treatment groups with respect to the baseline factors most important to determining the outcome, thus enabling more precise estimation of treatment effect. The stratamatch package extends this approach to the observational setting by implementing functions to separate an observational data set into strata and interrogate the quality of different stratification schemes. Once an acceptable stratification is found, treated and control individuals can be matched by propensity score within strata, thereby recapitulating the block-randomized trial design for the observational study. The stratification scheme implemented by stratamatch applies a "pilot design" approach (Aikens, Greaves, and Baiocchi 2019) to estimate a quantity called the prognostic score (Hansen 2008), which is used to divide individuals into strata. The potential benefits of such an approach are twofold. First, stratifying the data enables more computationally efficient matching of large data sets. Second, methodological studies suggest that using a prognostic score to inform the matching process increases the precision of the effect estimate and reduces sensitivity to bias from unmeasured confounding factors (Aikens et al. 2019; Leacy and Stuart 2014; Antonelli, Cefalu, Palmer, and Agniel 2018). A common mistake is to believe reserving more data for the analysis phase of a study is always better. Instead, the stratamatch approach suggests how clever use of data in the design phase of large studies can lead to major benefits in the robustness of the study conclusions.

A pilot design for observational studies: Using abundant data thoughtfully.

Observational studies often benefit from an abundance of observational units. This can lead to studies that-while challenged by issues of internal validity-have inferences derived from sample sizes substantially larger than randomized controlled trials. But is the information provided by an observational unit best used in the analysis phase? We propose the use of a "pilot design," in which observations are expended in the design phase of the study, and the posttreatment information from these observations is used to improve study design. In modern observational studies, which are data rich but control poor, pilot designs can be used to gain information about the structure of posttreatment variation. This information can then be used to improve instrumental variable designs, propensity score matching, doubly robust estimation, and other observational study designs. We illustrate one version of a pilot design, which aims to reduce within-set heterogeneity and improve performance in sensitivity analyses. This version of a pilot design expends observational units during the design phase to fit a prognostic model, avoiding concerns of overfitting. In addition, it enables the construction of "assignment-control plots," which visualize the relationship between propensity and prognostic scores. We first show some examples of these plots, then we demonstrate in a simulation setting how this alternative use of the observations can lead to gains in terms of both treatment effect estimation and sensitivity analyses of unobserved confounding.

OrderRex clinical user testing: a randomized trial of recommender system decision support on simulated cases.

OBJECTIVE: To assess usability and usefulness of a machine learning-based order recommender system applied to simulated clinical cases. MATERIALS AND METHODS: 43 physicians entered orders for 5 simulated clinical cases using a clinical order entry interface with or without access to a previously developed automated order recommender system. Cases were randomly allocated to the recommender system in a 3:2 ratio. A panel of clinicians scored whether the orders placed were clinically appropriate. Our primary outcome included the difference in clinical appropriateness scores. Secondary outcomes included total number of orders, case time, and survey responses. RESULTS: Clinical appropriateness scores per order were comparable for cases randomized to the order recommender system (mean difference -0.11 order per score, 95% CI: [-0.41, 0.20]). Physicians using the recommender placed more orders (median 16 vs 15 orders, incidence rate ratio 1.09, 95%CI: [1.01-1.17]). Case times were comparable with the recommender system. Order suggestions generated from the recommender system were more likely to match physician needs than standard manual search options. Physicians used recommender suggestions in 98% of available cases. Approximately 95% of participants agreed the system would be useful for their workflows. DISCUSSION: User testing with a simulated electronic medical record interface can assess the value of machine learning and clinical decision support tools for clinician usability and acceptance before live deployments. CONCLUSIONS: Clinicians can use and accept machine learned clinical order recommendations integrated into an electronic order entry interface in a simulated setting. The clinical appropriateness of orders entered was comparable even when supported by automated recommendations.

Ancient Rome: A genetic crossroads of Europe and the Mediterranean.

Ancient Rome was the capital of an empire of ~70 million inhabitants, but little is known about the genetics of ancient Romans. Here we present 127 genomes from 29 archaeological sites in and around Rome, spanning the past 12,000 years. We observe two major prehistoric ancestry transitions: one with the introduction of farming and another prior to the Iron Age. By the founding of Rome, the genetic composition of the region approximated that of modern Mediterranean populations. During the Imperial period, Rome's population received net immigration from the Near East, followed by an increase in genetic contributions from Europe. These ancestry shifts mirrored the geopolitical affiliations of Rome and were accompanied by marked interindividual diversity, reflecting gene flow from across the Mediterranean, Europe, and North Africa.

A Machine Learning Approach to Predicting the Stability of Inpatient Lab Test Results.

A primary focus for reducing waste in healthcare expenditure is identifying and discouraging unnecessary repeat lab tests. A machine learning model which could reliably predict low information lab tests could provide personalized, real-time predictions to discourage over-testing. To this end, we apply six standard machine learning algorithms to six years (2008-2014) of inpatient data from a tertiary academic center, to predict when the next measurement of a lab test is likely to be the "same" as the previous one. Out of 13 common inpatient lab tests selected for this analysis, several are predictably stable in many cases. This points to potential areas where machine learning approaches may identify and prevent unneeded testing before it occurs, and a methodological framework for how these tasks can be accomplished.

Signals of Variation in Human Mutation Rate at Multiple Levels of Sequence Context.

Our understanding of the human mutation rate helps us build evolutionary models and interpret patterns of genetic variation observed in human populations. Recent work indicates that the frequencies of specific polymorphism types have been elevated in Europe, and that many more, subtler signatures of global polymorphism variation may yet remain unidentified. Here, we present an analysis of the 1000 Genomes Project supported by analysis in the Simons Genome Diversity Panel, suggesting additional putative signatures of mutation rate variation across populations and the extent to which they are shaped by local sequence context. First, we compiled a list of the most significantly variable polymorphism types in a cross-continental statistical test. Clustering polymorphisms together, we observe three sets that showed distinct shared patterns of relative enrichment among ancestral populations, and we characterize each one of these putative "signatures" of polymorphism variation. For three of these signatures, we found that a single flanking base pair of sequence context was sufficient to determine the majority of enrichment or depletion of a polymorphism type. However, local genetic context up to 2-3 bp away contributes additional variability and may help to interpret a previously noted enrichment of certain polymorphism types in some East Asian groups. Moreover, considering broader local genetic context highlights patterns of polymorphism variation, which were not captured by previous approaches. Building our understanding of mutation rate in this way can help us to construct more accurate evolutionary models and better understand the mechanisms that underlie genetic change.

Systolic Blood Pressure and Risk of Type 2 Diabetes: A Mendelian Randomization Study.

Observational studies have shown that elevated systolic blood pressure (SBP) is associated with future onset of type 2 diabetes, but whether this association is causal is not known. We applied the Mendelian randomization framework to evaluate the causal hypothesis that elevated SBP increases risk for type 2 diabetes. We used 28 genetic variants associated with SBP and evaluated their impact on type 2 diabetes using a European-centric meta-analysis comprising 37,293 case and 125,686 control subjects. We found that elevation of SBP levels by 1 mmHg due to our genetic score was associated with a 2% increase in risk of type 2 diabetes (odds ratio 1.02, 95% CI 1.01-1.03, P = 9.05 × 10-5). To limit confounding, we constructed a second score based on 13 variants exclusively associated with SBP and found a similar increase in type 2 diabetes risk per 1 mmHg of genetic elevation in SBP (odds ratio 1.02, 95% CI 1.01-1.03, P = 1.48 × 10-3). Sensitivity analyses using multiple, alternative causal inference measures and simulation studies demonstrated consistent association, suggesting robustness of our primary observation. In line with previous reports from observational studies, we found that genetically elevated SBP was associated with increased risk for type 2 diabetes. Further work will be required to elucidate the biological mechanism and translational implications.